Introduction

Music is composed and performed for many purposes, ranging from aesthetic pleasure, religious or ceremonial purposes, or as an entertainment product for the marketplace. In the past 50 years, dramatic changes have taken place in music and the way music changes is worthy of investigating. This project mainly focus on the changes in the perspective of lyrics and is seperated into two parts:

Exploratory Data Analysis of Songs from 1970s to 2010s

  • Proportion of Each Genre in Different Decades
  • Overview of Lyrics Length by Different Types of Genres
  • Sentisment Analysis of Each Genre in Different Ages and Cluster Result based on Sentisment Analysis

Lyrics Style of Pop Singer Avril Lavigne

  • Sentisment Analysis of Lyrics of Avril Lavigne
  • Clusting and Topic Share of Songs of Avril Lavigne

Primary Methods Used

  • Text Mining
  • Sentiment Analysis
  • Topic Modeling

Proportion of Genres in Different Decades

The question of this section we want to answer is: Does the popularity of each genre change over the years? To answer this question, we plot the pie charts based on different decades ranging from 1970s to 2010s.

From the plots, we can observe that the numbers of each genre all raise gradually with a brust from 1990s to 2010s. The share of Rock music in music industry has declined while the popularity of Pop and Hip-Hop increase. But Rock music still dominate in the industry. Other conspicuous patterns are:

  • The consistency of Country and Mental
  • The emergence of Indie to music industry
  • The decrease of R&B

Overview of Lyrics Length by Different Types of Genres

The question of this section we want to answer is: How does the lyrics length of each genre vary with time changing. Two relevant plots are drawn to answer this question (We omit the situation of 2000s in first bunch of plot due to time-consuming running):

The length of each genre, on average, have actually increaed with time proceeding. Genre such as Country and Metal tends to raise, while Hip-hop, Folk tend to focus on catchy and short words. There are also genres of music remaining steady in word length. It is alse worth noting that the increase of lyrics length is not obvious on average.

Sensitive Analysis and Clustering

The questions we want to answer in this section are: How does sentiment released by lyrics change in each age with respect to overall music industry and what kinds of genres can be clustered based on previous sentiment analysis. In this section, we apply sentiment analysis using NRC sentiment lexion which is a list of English words and their associations with eight basic emotions (anger, fear, anticipation, trust, surprise, sadness, joy, and disgust) and two sentiments (negative and positive).

The emotions conveyed by lyrics perspective have been stable in the past 50 years. With the dominant emotions such as joy, anticipation and trust in lyrics, the existence of music tends to deliver more postive sentiment to audience than negative one. Based on the sentiment factor, we cluster different genres into groups and the plot is as follow:

Surprisingly, Metal music stays far away from the other genres, which arouses our attention to further deep into how emotions Metal conveys are diifferent from others. We plot a heatmap based on each value assigned to each genre according to the extent of one specific emotion included from sentiment analysis.

The heatmap reveals that Metal is a kind of music highlighting great passion on emotions. The value of each motion is much higher than those of other genres, especially the negative ones such as anger, disgust, fear and sadness. And this is the reason why Metal is so separate from other genres in clustering plot.

Lyrics Style of Pop Singer Avril Lavigne

This is the second part of the project. The reason why I pick Avril Lavigne is because she has been my favorite pop singer since I was in middle school. Meanwhile, the amount of lyrics data about her is sufficient for further research and analysis.

Sentisment Analysis of Lyrics of Avril Lavigne

In this section, we wannt to analyze the emotions and sentiment contained in the lyrics of Avril. By Wordclouds and the lexicon of Bing, we visualize the frequency of used words in her lyrics and their corresponding postive or negative label.

The visualization indicates that Avril uses more negative words than positive ones. Frequently used negative words such as damn, falling, hate and cry contribute to the third dominant emotions as sadness shown in the sentiment plot. This result matches with my personal experience when listening to her songs. Famous songs such as Everybody Hurts, Complicated, When You're Gone and Wish You Were Here all convey deep sadness or missing to the people loving as well as hate to the multifarious prejudice and secular rules outside world. Meanwhile, majority of the songs also reveal positive attitude such as faithful trust to the other half or firm desire to reconciliate to the surroundings towards unlucky situations, which implies why joy and trust are still two major emotions in Avril’s lyrics.

Clusting and Topic Share of Songs of Avril Lavigne

This section mainly clusters the songs of Avril Lavigne into diffferent groups based on 10 manual assigned topics. These topics are generated by the most popular terms and the most salient terms for each topic through LDA in Topic modeling. These topics are Lifestyle, Attitude, Love, Persistence, Personality, Relationship, Belief, Freedom, Hate and Sadness.

The two plots demonstrate how songs of Avril Lavigne are grouped based on 10 topics assigned and the share of each tpoic included in each song. The overlapping part in the cluster plot shows that some of songs imply high similarity from the persepective of topic. This situation may happen betweens songs from the same album.

Summary

This project carries on exploratory data analysis of songs from 1970s to 2010s and analyzes lyrics style of pop singer Avril Lavigne.

For the first part, we conclude that the proportion of genres varies from decades to decades. Two noticeable changes are the declining share of Rock in music industry and the raise of popularity of Pop and Hip-Hop. From the lyrics length perspective, the average length does increase but in a slight way. As to sentiments revealed by each genre, majority of them are postive. However, Mental separates itself from others due to more opposite emotions conveyed.

For the second part, my favorite Pop singer Avril Lavigne tends to use more negative words than the positive ones in her lyrics. But, optimistic attitude covered by seemingly negative words leads to more positive emotions dominating in her lyrics. Every singer has his or her own style to express the opinion about music. This style label is vividly shown in the clusting and topic share plot in Topic Modeling section.

Lyrics is indeed an important part of music. Much valuable information can be scratched by analyzing lyrics of each song. For a comprehensive understanding of a song or a genre, elements such as melody, tempo or even culture behind it alse play a essential role.